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'This paper discusses research ex-p±oring the < 
performance 'factors involved in studies whiph test for formal" 
opewtions.. It -is suggested that s^everal previous studies dealing- 
with formal operations 'confound performance and competence in * ' , 
determining the^ -extent of subjects^ formal operation^ schemata.. 
^Factors' which might account for performance differences are: (1) - 
tasks chosen by the investigators to measure formal operations 
thought^ (2) the degree .to which the investigators use a. direct 
method of task presentation and- questioning^ (3) the number of 
dimensions along which tasks varyV '(^) ^he ccJntent area of^'the task/ 
(5.) sex differences^ ,and (6) intelligence differences. Hesults of a^- 
series of expeiu-ments with elementary ^ junior high^ and, high school 
students which tested the extent to which these factors affect formal 
operations performance are presented, (BJT)^ 
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^ * T y STATED f 

• . SENT Of f 

Tormal Operations Thinkings Now You Se6 It, No« You Don't 

,00 ' " r I . .... 

'xT^ ^ • Sus^rin^ Martoraao 

^jj TKe title of Ehis^aper reffects my alternating stages of confidence when 

trying to understand 'my own rese^rra, and that of others, on fortnal operations 
thought". As any one of my^/es'^arch assistants will tell you, I cycle from 
confidence to uncertainty, Ij^ -^^g^P^^^r finally to renewed hope before I 
start ov^r again. .1 might ado that the period of confi:dence is siArely the 



briefest part of t^e cygle 



In general th%^ m^ds are Highly correlated with the amount of time I have 
► sp^nt tfhe previous 'BajT t;r^ing to decipher my own data and integrate it with the '' 
results of jpthe'r studies. vLf we take ,as our ^a^ndard^Inhelder and Piaget's (1958) 

'Statgments^about thq character o^^'fotmal thought and its course of development, 

* " • ■* / ■ 

th^n the research literature is clearly equivocal* in its support. Although 

' • . ' 

early invet^tigbtbrs (Jackson, 1965; Lovell, 1961), in agreement* with Inhelder 
Bnd Piaget , '*Q4?t the age of "Emergence of formal operation^ at betw^n 11. and 12, 
more recent studies (Dale, 19^0; Dulit, 1972; Keating, 1975; Martc^^o, 197^; 
N/7\* Weybright, 1972) have* failed to fic>d a majority of ^ even middle and late adolescent 



suB^ects perroriqing at a/ formal level . Interpretation of these data Is especially 



difficult^ belcause the range of within subject performance across^ tasks also 
"varies^etw^en ^i^Tes. ' While JKeating (1975), Lee (1971) and Weybright (1972) 
have found With^in subj«.ct homogeneity of performance across tasks, I (Martor^lno, 
1973) and dthers (Dulit, 1972; Kuhn, Langer, Kohlberg & Haan, 1972; Lovell, 1961; 



L9velL, 1971; Neimark, 1970) have found within subject dif f erences acrps^ tasks. 
j^Xj^ . I would lik6 to suggest that the performance - competence distinction pre- 
sented by Flave^l andWbhiwill (1969) is a useful conceptual strategy for 



, <,clarifying the research on' cognitive development at the formal operations level. 
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Inhelder and Piaget's (1958) statement about the development and nature of 
y formal operations is essentially a competence theory . That* is, a forftial or^^ 

logical presentation of the structure of formal thought with a description of 
' what the individual knows and does in an ideal environment. Even a^brief , 
reading of the Inhelder and Piaget (1958) book will^reveal that the authors 
" ' are di-Siy^ing the logical structure"*" of adolescent thinking. But, how then 
do we deal with the Empirical data presented in the numerous subject •(>rot0fcols 
included in the book? Dulit (1972) has provided the answer by pointing out 
' that the subject protocols \^ere chosen not for their representativeness of 
subject'^ performance in general, but for theif value in illustrating the authors 
explanation of. the chauacter'of formal thought as revealed in performance on 
^the formal operations tasks. Thus, I would argue that Inhelder and Piaget 

are describijag competence, and therein lies the problem fj>r those for us who 
. do formal opp^atiocis research, T y • 

Despite the use of the clinical method when testing subjects, I believe 

• • ■ 

the formaj. operations data reflect an interaction between performance and 
competence. In discuasing performance aspects of behaviour I am referring 
to those processes and conditions in real problem situations that facilitate \ 
pr inhibit the use of the knowledge implied by a .given level of competence. 
A per|ormance theory therefore is concerrted'^vjI^h^Bpecifying the conditions or 
strategies th&t facilitate or impede performanbe. - ^ ' 

I'd like to turn now to Sn examination of some of the dittferences in 
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previous formal operations studies that might account for the performance 
differences that I mentioned earlier. Perhaps the most striking difference 
between studies is the (gc>ual task or tasks chosen by the investigator feo ^ 
mejasure formal thought. Inhelder and Piaget (1958b -imply that the formal 
operations schemata emerge synchronously and thus terform'ance 'on tasks de- ^ 
signed to elicit these schemata should be equivalent within a given subject. 
However, my doctoral research (MaVtorano, 1973) showj/d^cotisiderable heterogeneity 



across a wide range of the tasks. Whlle^it^is n&t possible to determine if 
this task performattce difference represents differential competen^^with re- 
spect t*o the underlying schemata tapped; or differential task difficulty, I 
think it is safe to say that the task^an inve&tlgatlor uses wilxSiave an effect 

> : V J); 

on whether or not evidence of formal thinking^ is found. 

•Another factor that seems to vary between studies is the degtee to which , 
the investigator uses ^ directed method of task presentation and ques^j^^^-^Sf 
It i8 difficult to determine how great the variability is on this dimension, 
but a clooe reading of the methods sections of a number of studies leads me 
to suspect that there are disc/riminable differences. For instance, Lee (1971) 
utilized a very specific method of task presentation that involved a series of 
choice points, if the subject did not show eVidence of a certain level of 
understanding, then the interview was terminated. I'm sure one could argue 
that this wa either a useful procedure (the choice points proceeded from the 
<rmost simple instance-s 6f the concept to the most general) or ^hat the procedure 
did. not adequately alloJ^the subject to explore all the ramifications of the 
problem apparatus. The point is still the same - the procedure may 'Have 
affected subjects' performance in a way that was different from th^t in other 
studies where the task procedure was different, ^ variation on this point is 
the distinction between use of thk^6lual task- apj^^atus described by Inhelder 
and Piaget (1958), or^a pencit^a^ paper version^^^^>5^^ used a pencil 

and paper version of the chem^alls task and argue^^that if subjects were at a 
formal level of thought they us^d a system either idhen combinijafg the actual 
chemicals, or when simply noting which combinations were made.t However, the 
same would not be expected of transitional -subjects. FlaveU and Wohlwill -(1969) 
and Neimark^i (1972) have pointed out that transitional subjeqts m^ be especially 
susceptible to situational variables and thus very different re^sults might be 
^kpected in these two versions of the task. • 

000.04 



The number of dimensions along which a jjpsk varies' may al^ affect performance. 
For instance, the pendulum task has tvo very obvious dimensions, amount of weight 
and length of the^ndulum, while the rods problem has five; length, shape, 
material and diameter of tihe rod and amount of weight attached. Similarly, Piaget 
^nd Inhelder (1951) note- that young children can systematically^ contruct all 
possible pairs of three colors but become confused wh||| more colors are added. 

The content area in which ^le^task is set is the final tasic variable that 

I would like to mention, Piaget (1972) himself has suggested that evidenci^ of 

formal thought may appear first in those content areas In Vhich the subject; has 

special interest or aptitude. I suspect that area of specialization is a per* 

formance variable and that it will interact with the subject's level of cognitive 

competence. Those subjects who have an integrated formal structure will be able 

to demonstrate an* understanding of the operational schemata in a number of different 
♦ 

content areas, while transitional subjects will be limited to an understanding 
of the concept only in those content areas in which they have special interests 
and aptitudes. 

/ 

A second class of performance variables relates to characteristics of the - 
subject. I will- brief ly mention two that^h^ve been reported in the research 
litqrature: Sex differences and intelligence. 

Sex differences, when fou^ (Dale, 1970; Dulit, 1972), indicate, that males / 
score higher than females. However, this is an elusive result and might be ex- 
pected to vary as a function of the content area in which the task is set. To 
date no one has offered an explanation of why cex differences might occur, ^br 
presented evidence to support their hypothesis. 

J 'three studies have examined the effect of hi^ IQ ort ^of mal operations pet- 
formance*. Two^studies (Dulit, i972; Keating, 19Z5rhave found that high IQ is 
associated with highel^^^fo^ip^^ scorers when comparecl| to p^fprmance of 

same age subjects of average IQ. Webb (1974) howeve/, failed to find'IQ * 



differences. Certainly this variable should be controlled in future rese|ft:ch. 
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Tq summarize, I have suggested that Inhelder and Piaget's (1958) description 
of the development of, formal operations thinking is a competence theory. The 
data derived from research on formal operations reflects an interaction between 
the subject's underlying competence and a number of unspecified performance 

factors, ^ ^ 

I would ndw like to present the results of some research I have been doing 
on the feet s- of performance factors in formal operations task situations. The 
fim study was simply an J^ffort to replicate a portion of 'the results of my^ 
doctoral dissertation. In my dissertation I had given ten of, the formal opera^ 
tions tasks to a fairly wide age range of junior high+school and „high school 
girls. While I found a significant main effect for age, of * greater interest 
was the significant main effect for tasks. Withirf subject performance across 
the ten- tasks was not homogeneous, and in fact, there' was a predictable sequence 
for the ten tasks. Those tasks that tapped tite combinatorial and correlations 
schemata elicited a higher level of performance lhaft, did those tasks which 
tapped the schemata of multiplicative'' compensation, proport'ionality , and 
mechanical equilibrium. There were also differences' between these other 
schemata,* With multiplicative compensation and proportionality eliciting 
higher "levels of performance th^ the tasks , tapping mechanical equilibrium. 

In this first study, I tested 11-, 12-, and 13-year oFd children ^on the 
colored tokens, permutations, and correlations tas^s The ANOVA of the data ^ 
indicated that age was significant at the .05 level ^(F = 3.75, df = 2, 48),' 
with the 12-year old subjects performing^ a higher level than either the 
11- or 13 -year old subjects. The main effect for taskD wafl aloo highly 
significant (F ° 1^.11}^. 2, 96, p = .001),. while the remaining main 

effect, OCX, wajs not significant'. In agreement with my doctoral data, the 
• 'dOlorpd' tokens eaok elicited significantly higher levels of performance than. 
dW eithep^he ^ipxAtatttViS or "correlations task's. There was/no significant 
difference betwejm the permutations and the correlations tasks. Th^ sex by ^' 



task Interaction was significant atiU .05 leAiel (F = 3.24, df = 2, 96) and^if/^- 

^ . \ ^ - . 

due to a crossx)ver effect, with male subjects p/erforming better on the correlations 

tasjc, and. female subjects, performing ^tter on, the permujpati-ons task. Both 

8e:j;(^i performed similarly ^he^colbVed tokens Jrask. ^ * 

The -resul'ts of thi^ study gave me more confidence in my dissertation datti 

and further irtcrea^sed my curiosity about, the colored tokens task, .^n this 

sCud^ 733!, of' /the 11- and 13-year old subjec^ts and- 83% of the 12-year old . 

subjects per forth^d at ^an early or late formal operations level on the colored 

tokens teTsk. ^ThilT'Tagk, like the. permutations and chemicals tasks is described 

# ' * " . ^ " ... 

^by Inhelder and' Pipage ^1958^ asrirappin^^ an unde^^anding of the combinatorial 

system: ^He task Jjas^ fr^quiiltly been used as ^n(lnaicator of an individuals 

ability to engag4:.in fot^l ot)erations thought. However, if my data are accurate,' 

performance' on tWis task is very^di^ferent from performance on arfother task - 

permutations - that requires understanding of the same logical concept. In 

addition/ the data from the colored tokens task indicated that formal opera- 
* ' 0* ^ , ■ • ' - 

• tions thoueht was present in a majority of early adolescent Subjects, These- 

'results ire clearly inconsistent \gith a number of the other studies on formal 
operations tha^ I mentioned earliefr. | * 

. -I then deqideTlo examine the colored tokens task more fully. As described 
by Goodnow (1962), the colored tokens task involves presentation of piles of 

" V ' • - ■ 

six di;Eferent colored chips dr squares of paper. The subject is instructured 

• ' \ 

to make ^11^ the p'dirs of "colorS/hat he can think of wljthput any repeats. The ^ 

child is theti allowed to lay out the different pairs of colors on tKe table in 

front 'ofjriift. Seroing of the task lo based on whether the pairs are made. 

randomly, in conformity with a perceptual scheme, or in a systematic way that 

insures lhat all possible pairs will be found. A late formal operations score 
. ■ ' ~^ 

y.. implies that the subjec't. has a systematic strategy, and that he can state a 

0 ( ^ 0 ' ■ 

genesl^strategy ¥or making all pairs when a different number of colors is ' 
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In thinkirxg aSouftfthe colored tokens task and the high percentage of formal 
scores orbtained by ^oungaH^escents, I reasoned, that perhaps the use o*f colored 
squgre^/o^E p-apeV was a particulary apt choice of stimulus materials. Children 
\ave a great deal of experience with colors ^rom the beginning of their school 
I careers 6n, and perhaps) the early emergence of formal thinking on »this task was 
du^'tfo the subjects^ high degree of f ajniliarity with, the materials. Consequentljt, 
I decided to simply .change the stimulup materials initially. 

Unfortunately, tl>itt.gs are* Aever souj^simple; By the time I had thought of 

• . . ^ • ., ; .... ■ .J . 

'this study I had already exhaus^ted my si^pply of subjects at the college campus 

-*. 

school, and the schobi yea.r was drawing quickly to a close* So I improvised, 
.< * . " ' * ^ ' . 

* I had already tested all of the 9- and 10-year old children on the colored 

' tokens task early^ in th/ fall and* I kne^^ that a*"* number of^he chit^en 'Vere 

in a transitipnal phase' on the problem. \ decided fo i*etest. these children ^ 

■ <w>" . '* ^ • • - 

dhd include the stimulus materials factor in my desiign. Tbe subjects vere 

- ■ J. ^ , ' ' r ^ * ^ V „ . 

already categorized by age (9- vs. 10-years) , -sex, and classroom. Half of 
- ' the: streets in each group received the otig^al Colored tokens task the 
secoJd time, while the' other half received thV'^same J)r6blem 6ut with new 
mater^alsf Pictures of six different fruits apd a cover sto^y about, ifeking 
^ up new fl^ors^f jello. * ^Inst^a^-^ asking the child in this new condition 

. to make all the possible pairs of colors, he/she i?as asked to miflce all the ^5 
po88lble;,.^p^irs of fruit flavors.. \ ^ 

Th^ ANOVA tested fdr the effects of oeat^.'age, time at which the task was 
' given (fall or spriiflgO, and whether the task, givej^ in the spring usgd the) same 
\ materialG as the ^ask giv.en in the. fall ^ new materlalg. Tho^only effect that 
was significant, and it was significant at^'beyond the .001 level, was the y.me 
" at which th^ to^k had been given (P.= 24.80, df ^c=3t^0).. In the fall 29^ of 
the children, had performed at either an early or late formal operations level 
V ^ on the task^ in contrast to a preoperations or concrete level. In the firing, 
Q * 587. of the children who received ttie same task a secofifl time, and 54% of the 

ERIC , : ' - ' 

-— ■ /, ■ y ,. ■■ , ,00008 , - • • . 



children who recei\jed the fruits variation scored at an early or late formal 

* .•• • *^ 

level. Of course it is impossible to know what experiencffs these children had 

" ■' • .' 

during the 7 months that intervened betwelen tjie first and second testings, 

** * 
hoiiever I do know th^^t the children did not haVe any direct instruction on 

this concept. / ... ^ J, 

^ I think one important result of ^:his study is the fact that there was no 

difference between ^the two stimulus conditions at the second time of testing. 

i • 

In ^effect, the subiects' understanding of the concept of combtpations improved 
regardless of whether the second tasU was new or the same. The result is even 
more powerful in light of the fa^t that 66% of the children perfprmed at a 
preoperations or earl^ concrete level on the initial presentat'lon of the problem. 
The second important finding was 'the fact th^ there was such a dramatic improve- 
' ment in performance-^ a group •of s^jects ^lo initially demonstrated a very low 
level of performance. '\ 

The final study that I would like to describe carried my examination of 
^ task factors further. Once again I used 9- and lO-year old subjects and the 
calored tokens task. In an effort to. check that stimulus material's did not 
have an effect if the subject had never received the colored tokens previously, 
I presented one group of subjects with the coJ.ored tokens task, one group with 

the fruits task that I used ^earlier, and one g'roup with a variation on the * 

■ . ' • . . ^ '\ ' '' \ 

fruits task. Two things were changed £h this.new task: The instructions, i 
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and th^ perceptual cues surrounding the layout of tfie task materials ^, The 
( instructions were changed to a statement that different flavors of jello 
could be made by raiKing these five flavors (strawberry, cherry, etc.) to- 
* gether and the task w'as^to make all the possible new flavors^ of jeilo. Thus, 
the child was not given any specific instructions to make all pairs, triads, 
and so forth. These new instructions turn the problem into an analogue of the 
Inhelder and Piaget (1936^ chemicals problem. In an effort to^educe th^^actual 
number of mixtures that the child would have to genel^ate, only five l^nds of* 
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fruits were used. 'The second change in the task,- the perceptual cues - Involved 
laying the piles of five fruit? in a circle irather in a^row as has been the case , 
in previous" studies. ^ ^ ^• 

Once again, the ANbVA revealed no main effect for the age varia^l^ Set 
was significant (F = 7.87, df « t, 60," p =' .01) -with males performing at,^ - ^ 
higher level than females.! The' task main effect was ^Iso significant (F = 4.60, 
df = 2, 60,Np .= .02K ' Once again, (k>l^red tokens performance was n<^ significantly 
different from the fruits problem ^^n -which only the stimulus- materials were changed. 
^Both tasks produced signif icantl^r higher-levels of performance than did the fruits 
task^that varied instructions, and perceptual cues* None of the ANOVA interactions 

were significant. , ' ' ' 

yprty-six percent of the subjects whd receive^ tHe^olored tokens * task per- 
formed at early or late formal operations aqd 427. of tjtie subjects who received ^ 
the simple fruits task performed at early or late fojrni'al operations. However, 
only 177o of the suj^jects who received the complex fruits'' task (ie. both • ItistTnc-^ 
tions and perceptual cues varied) scored at an early or late fon^at c^erationff, 
level. Two factors make this result even more impressive. In the complex 
fruits task only lEive different fruits w^re presented while the colored tokens 
anjd simple fruits tasks r&qu/red the silbjects to make P||^s from an initial 
base of six fruits or colors. Secondly^ the .same s-cofing criteria used for ^ 
the other two tasks were applied to the compl'^x task. , That^i-sf if a ri'ubject 
•systematically made all the possible palr.s of the -five fruits , he/she received 
a score of early foVmal operations. Actually, the task presented asked the 
Gub^J-eG:t ^to make all pooGible three-way mixtures, fou\-way mixtures^ and the 
mixture of atX five fruits together in addition to all the pairs of fruits. 
My^ decision to use the same scoring criteria was based on the fact th^t only 

0 

.7 .of the 24 sub jects 'who received the complex ftuits problem produced anything 

other than pairs of fruits and th^T'^y only produced a small^number.of random 
three-way mixtures. , ' 
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I am fairly confident that this striking result is not dtle to the sirafJle 
fact that the .in^structions were too ambiguous. I have some pilot data of college 
students ^* performance on the same task, and in' all cases* the student^ produced^ 
three-way, four-way, and the five-way mixtures in addition to the pairs* 

These results seem to clearly argue that young adolescents are not very 
systematic at all in using a corajjinatorial schema unless you specifically tell 
them to do so. Further, they db.not seem to spontaneously think of more than 
twp e-lements varying at a^ time. 

Let me quickly integrate my "introductory section with the results of 
these studies. I have suggested that there are a number of performance factors 

ttve in the experimental situations in which we have Rested for formal opera- 

I ■ ' \ 

s thinking. In most cases these factors have not be'en specified and we 

presently have' little understanding of how they affect the underlying competence 

6 

jEor formal operations thought that the subject brings to th^ experimental setting, 
The results of the studies I have presented suggest that the tasks used to* test 
formal operations. thinking elicit different levels of performance. Secondly, 
that while stimulus materials per se do not seem to affect performance, the 
amount ot structure given to the subject in the instiructions about what he is 
to do j))an^the .perceptual conf iguratioa of the stimulus materials do affect 
performance. In order to go beyond our present normative studies, we need to 
develop more standard procedure^and adequate controls for the performance- 4^ 
factors that affect the manifestation of the formal properties of "adolescent 
thought . . ' • * 
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